Search for: All records

We investigate knowledge retrieval with multi-modal queries, i.e. queries containing information split across image and text inputs, a challenging task that differs from previous work on cross-modal retrieval. We curate a new dataset called ReMuQ for benchmarking progress on this task. ReMuQ requires a system to retrieve knowledge from a large corpus by integrating contents from both text and image queries. We introduce a retriever model “ReViz” that can directly process input text and images to retrieve relevant knowledge in an end-to-end fashion without being dependent on intermediate modules such as object detectors or caption generators. We introduce a new pretraining task that is effective for learning knowledge retrieval with multimodal queries and also improves performance on downstream tasks. We demonstrate superior performance in retrieval on two datasets (ReMuQ and OK-VQA) under zero-shot settings as well as further improvements when finetuned on these datasets. 

Bacterial meningitis, an infection of the membranes (meninges) and cerebrospinal fluid (CSF) surrounding the brain and spinal cord, is one of the major causes of death and disability worldwide. Higher case-fatality rates and short survival times have been reported in developing countries. Hence, a quick, straightforward, and low-cost approach is in great demand for the diagnosis of meningitis. In this research, a microfluidic fully paper-based analytical device (μFPAD) integrated with loop-mediated isothermal amplification (LAMP) and ssDNA-functionalized graphene oxide (GO) nano-biosensors was developed for the first time for a simple, rapid, low-cost, and quantitative detection of the main meningitis-causing bacteria, Neisseria meningitidis ( N. meningitidis ). The results can be successfully read within 1 hour with the limit of detection (LOD) of 6 DNA copies per detection zone. This paper device also offers versatile functions by providing a qualitative diagnostic analysis ( i.e. , a yes or no answer), confirmatory testing, and quantitative analysis. These features make the presented μFPAD capable of a simple, highly sensitive, and specific diagnosis of N. meningitis . Furthermore, this microfluidic approach has great potential in the rapid detection of a wide variety of different other pathogens in low-resource settings. 

Water systems often contain complex macromolecular systems that absorb light. In marine environments, these light absorbing components are often at the air–water interface and can participate in the chemistry of the atmosphere in ways that are poorly understood. Understanding the photochemistry and photophysics of these systems represents a major challenge since their composition and structures are not unique. In this study, we present a successful microscopic model of this light absorbing macromolecular species termed “marine derived chromophoric dissolved organic matter” or “m-CDOM” in water. The approach taken involves molecular dynamics simulations in the ground state using on the fly Density Functional Tight-Binding (DFTB) electronic structure theory; Time Dependent DFTB (TD-DFTB) calculations of excited states, and experimental measurements of the optical absorption spectra in aqueous solution. The theoretical hydrated model shows key features seen in the experimental data for a collected m-CDOM sample. As will be discussed, insights from the model are: (i) the low-energy A-band (at 410 nm) is due to the carbon chains combined with the diol- and the oxy-groups present in the structure; (ii) the weak B-band (at 320–360 nm) appears due to the contribution of the ionized speciated form of m-CDOM; and (iii) the higher-energy C-band (at 280 nm) is due to the two fused ring system. Thus, this is a two-speciated formed model. Although a relatively simple system, these calculations represent an important step in understanding light absorbing compounds found in nature and the search for other microscopic models of related materials remains of major interest. 

Knowledge-based visual question answering (VQA) requires answering questions with external knowledge in addition to the content of images. One dataset that is mostly used in evaluating knowledge-based VQA is OK-VQA, but it lacks a gold standard knowledge corpus for retrieval. Existing work leverage different knowledge bases (e.g., ConceptNet and Wikipedia) to obtain external knowledge. Because of varying knowledge bases, it is hard to fairly compare models’ performance. To address this issue, we collect a natural language knowledge base that can be used for any VQA system. Moreover, we propose a Visual Retriever-Reader pipeline to approach knowledge-based VQA. The visual retriever aims to retrieve relevant knowledge, and the visual reader seeks to predict answers based on given knowledge. We introduce various ways to retrieve knowledge using text and images and two reader styles: classification and extraction. Both the retriever and reader are trained with weak supervision. Our experimental results show that a good retriever can significantly improve the reader’s performance on the OK-VQA challenge. 

GQA (CITATION) is a dataset for real-world visual reasoning and compositional question answering. We found that many answers predicted by the best vision-language models on the GQA dataset do not match the ground-truth answer but still are semantically meaningful and correct in the given context. In fact, this is the case with most existing visual question answering (VQA) datasets where they assume only one ground-truth answer for each question. We propose Alternative Answer Sets (AAS) of ground-truth answers to address this limitation, which is created automatically using off-the-shelf NLP tools. We introduce a semantic metric based on AAS and modify top VQA solvers to support multiple plausible answers for a question. We implement this approach on the GQA dataset and show the performance improvements. 

The partitioning of medium-chain fatty acid surfactants such as nonanoic acid (NA) between the bulk phase and the air/water interface is of interest to a number of fields including marine and atmospheric chemistry. However, questions remain about the behavior of these molecules, the contributions of various relevant chemical equilibria, and the impact of pH, salt and bulk surfactant concentrations. In this study, the surface adsorption of nonanoic acid and its conjugate base is quantitatively investigated at various pH values, surfactant concentrations and the presence of salts. Surface concentrations of protonated and deprotonated species are dictated by surface-bulk equilibria which can be calculated from thermodynamic considerations. Notably we conclude that the surface dissociation constant of soluble surfactants cannot be directly obtained from these experimental measurements, however, we show that molecular dynamics (MD) simulation methods, such as free energy perturbation (FEP), can be used to calculate the surface acid dissociation constant relative to that in the bulk. These simulations show that nonanoic acid is less acidic at the surface compared to in the bulk solution with a p K a shift of 1.1 ± 0.6, yielding a predicted surface p K a of 5.9 ± 0.6. A thermodynamic cycle for nonanoic acid and its conjugate base between the air/water interface and the bulk phase can therefore be established. Furthermore, the effect of salts, namely NaCl, on the surface activity of protonated and deprotonated forms of nonanoic acid is also examined. Interestingly, salts cause both a decrease in the bulk p K a of nonanoic acid and a stabilization of both the protonated and deprotonated forms at the surface. Overall, these results suggest that the deprotonated medium-chain fatty acids under ocean conditions can also be present within the sea surface microlayer (SSML) present at the ocean/atmosphere interface due to the stabilization effect of the salts in the ocean. This allows the transfer of these species into sea spray aerosols (SSAs). More generally, we present a framework with which the behavior of partially soluble species at the air/water interface can be predicted from surface adsorption models and the surface p K a can be predicted from MD simulations. 

The absorption spectra of molecular organic chromophores in aqueous media are of considerable importance in environmental chemistry. In this work, the UV-vis spectra of benzoic acid (BA), the simplest aromatic carboxylic acid, in aqueous solutions at varying pH and in the presence of salts are measured experimentally. The solutions of different pH provide insights into the contributions from both the non-dissociated acid molecule and the deprotonated anionic species. The microscopic interpretation of these spectra is then provided by quantum chemical calculations for small cluster models of benzoic species (benzoic acid and benzoate anion) with water molecules. Calculations of the UV-vis absorbance spectra are then carried out for different clusters such as C 6 H 5 COOH·(H 2 O) n and C 6 H 5 COO − ·(H 2 O) n , where n = 0–8. The following main conclusions from these calculations and the comparison to experimental results can be made: (i) the small water cluster yields good quantitative agreement with observed solution experiments; (ii) the main peak position is found to be very similar at different levels of theory and is in excellent agreement with the experimental value, however, a weaker feature about 1 eV to lower energy (red shift) of the main peak is correctly reproduced only by using high level of theory, such as Algebraic Diagrammatic Construction (ADC); (iii) dissociation of the BA into ions is found to occur with a minimum of water molecules of n = 8; (iv) the deprotonation of BA has an influence on the computed spectrum and the energetics of the lowest energy electronic transitions; (v) the effect of the water on the spectra is much larger for the deprotonated species than for the non-dissociated acid. It was found that to reproduce experimental spectrum at pH 8.0, additional continuum representation for the extended solvent environment must be included in combination with explicit solvent molecules ( n ≥ 3); (vi) salts (NaCl and CaCl 2 ) have minimal effect on the absorption spectrum and; (vii) experimental results showed that B-band of neutral BA is not sensitive to the solvent effects whereas the effect of the water on the C-band is significant. The water effects blue-shift this band up to ∼0.2 eV. Overall, the results demonstrate the ability to further our understanding of the microscopic interpretation of the electronic structure and absorption spectra of BA in aqueous media through calculations restricted to small cluster models.